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Introduction 

Skills in several foreign languages are among the core competencies demanded in today’s 
multicultural, mobile and connected society. The desired skills, however, imply more than learning and 
mastering abstract linguistic tasks related to a language’s structure or vocabulary. Rather, the objective 
is to develop communicative competence, meaning language learners should acquire the skills 
necessary to communicate appropriately with people from a variety of cultures. In other words, 
foreign language teaching must also promote the development of intercultural competence 
(henceforth IC) and adaptability. Indeed, more recent curricula explicitly require intercultural 
outcomes (such as intercultural sensitivity, tolerance and openness) in addition to the standard, 
language-based skills (Council of Europe, 1982; D-EDK, 2013; Ingram & O’Neill, 1999). 

While expanding students’ IC, and thus making a substantial contribution to a cooperative 
coexistence in an increasingly globalised world, is an established goal in foreign language teaching, it 
is also clear that foreign language lessons per se will not always meet this goal if they do not actively 
work towards it and if learning opportunities in this area are lacking (Ingram & O’Neill, 1999, p. 3; 
Weier, 2009, p. 10). Contact with speakers of the target language (henceforth TL) and representatives 
of the target culture in the form of study abroad or exchange programs provides such opportunities 
(Weier, 2009, pp. 10-11). Such programs are often an organizational and financial challenge both for 
schools and students. It is all the more important, therefore, to evaluate the impact of these programs 
with respect to different goals (academic goals, foreign language goals, intercultural goals) and to 
identify key factors contributing to fruitful implementations (Anderson & Lawton, 2011; Engle & 
Engle, 2004; Tarrant, et al., 2014). This requires longitudinal large-scale research including a control 
group that investigates a broad array of study abroad / exchange programs that differ with regard to 
some key variables (such as duration, age of participants, accommodation, amount of preparation, 
etc.). Such studies are still rare up to this date (Anderson & Lawton, 2011; Tarrant, et al., 2014) and 
usually work with rather small samples. Furthermore, research on the impact of study abroad on 
intercultural learning also has to shed some light on the sustainability of study abroad / exchange 
experiences. Unfortunately, there are very few studies that include a delayed post-test (for exceptions, 
see Dywer, 2004; Rexeisen, et al., 2008). 

Literature Review 
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Although the majority of experts consider contact with TL speakers to be an ideal means to 
positively influence attitudes and skills in intercultural issues, research on the effects of intercultural 
contact or study abroad programs has revealed that contact alone is not sufficient to attain positive 
results. Researchers agree that study abroad programs only contribute to intercultural learning if the 
students really ‘experience’ the host culture by truly engaging with host country participants while 
working or studying (Anderson & Lawton, 2011, p. 87; Donnelly-Smith, 2009). From the influential 
contact hypothesis (Allport, 1954) we know that the contact situation must fulfil certain criteria to 
realistically achieve positive effects in terms of intercultural attitudes. For instance, the individuals 
involved should share similar interests and have about equal social status. The contact situation should 
also allow for interaction and should be endorsed by a person of authority (Alexander & Perl, 2010, 
pp. 288, 291; Baker, 1992, pp. 107-108; Dornyei & Csizer, 2005, pp. 328-329; Flewstone, 1996, p. 327; 
Leyens, et al., 1994, p. 47; Ogay, 2000, p. 274). 

Gardner (1985, p. 85), for instance, found that a four-day excursion to Quebec significantly 
influenced the intercultural attitudes of Anglophone students and led to an increased interest in 
learning French. At the same time he points out, however, that interaction alone is not enough to 
positively influence attitudes, that the kind and amount of interaction are crucially important 
(Gardner, 1985, p. 86). This is also suggested by a study by Thomas which indicates that 
accommodation in a host family and joint project work with representatives of the TL improves the 
intercultural learning (Alexander & Perl, 2010, pp. 290-291). 

In addition to the above-mentioned factors, the duration of the exposure and the motivational 
and attitudinal starting conditions of the people involved play a role (Engle & Engle, 2003, p. 9). As 
far as the ideal length of stay is concerned there are mixed research results. Research suggests that 
longer programs are associated with greater gains in IC (Dwyer, 2004; Engle & Engle, 2004; Kehl & 
Morris, 2008; Medina-Lopez-Portillo, 2004). Nevertheless, short-term programs have also been shown 
to have a positive impact on cross-cultural development (Anderson, et al., 2006; Dwyer, 2004; Chieffo 
& Griffith, 2004; Donnelly-Smith, 2009). There are even studies which suggest that short excursions, 
in particular, have the potential to positively affect intercultural attitudes and that a more extended 
contact with the target group can lead to a renewed deterioration of intercultural attitudes (Gardner 
1985, pp. 88, 106; Dornyei & Csizer, 2005, pp. 351-352). This may be attributable to the somewhat 
superficial nature of the cultural immersion in short excursions which may prevent participants from 
encountering intercultural difficulties or culture shock. In the words of Engle & Engle (2003, p. 6) 
such excursions provide new and interesting scenery but not a new environment which is characterized 
by interaction. As Ogay (2000, pp. 274-277) points out, it is possible that short-term study abroad 
programs only yield positive effects at first sight and do not really advance intercultural learning since 
contact with representatives of the other cultural group is not intensive and manifold enough for 
participants to recognize differences or to encounter intercultural challenges and handle them. It is 
exactly the culture shock or the experience of unfamiliarity and difficulties that forces exchange 
students to grow interculturally. As Engle & Engle (2003, p. 5) put it “gain only comes at the expense 
of a certain pain”. 

Research on the influence of longer study abroad programs on the intercultural attitudes of 
exchange students suggests a different trajectory of attitudinal development. Often study abroad 
participants have particularly positive intercultural attitudes before travelling to the host country. Once 


©2015 The Forum on Education Abroad 


188 



Frontiers: The Interdisciplinary Journal of Study Abroad 


Volume XXV, Fall 2015 


they live in the host country, these attitudes tend to decline. Provided they stay in the host country 
long enough, their intercultural attitudes become more favourable again (Gardner, 1985, pp. 88, 106; 
Dornyei & Csizer, 2005, pp. 351-352). Such a U-shaped development could potentially be explained 
with differing starting conditions of the participants. In the studies by Gardner (1985) and Dornyei & 
Csizer (2005) which focus on short-term contact, the contact with representatives of the host culture 
was not freely chosen by the participants. Hence, it is likely that not all the participants were positively 
disposed towards the pending intercultural exchange and the other cultural group. In the case of long¬ 
term study abroad programs, the linguistic and cultural exchange is normally chosen by the 
participants themselves. It is likely that the intercultural attitudes of these students are positive before 
they depart. Consequently, it is also more difficult to find positive changes. It may also be that 
intercultural attitudes are so positive before the actual exchange that they cannot be upheld once the 
students are confronted with linguistic and cultural challenges in the host country, and that, therefore, 
cross-cultural attitudes may become less favourable in the beginning of the program. 

That the starting conditions of the students play a role in the development of IC is underscored 
by a study by Hammer (2005) on the short- and long-term effects of AFS activities. The study suggests 
that interculturally less competent adolescents can profit more from their stay abroad than those who 
are already interculturally relatively competent before their stay abroad. 

While positive effects do not happen automatically as a result of contact but are dependent on a 
number of factors, such as length of program, amount and kind of contact with representatives of 
the target culture, social status of the involved groups, the affective starting conditions of the 
participants, etc., studies comparing the intercultural development of study abroad students with that 
of a control group convincingly show that study abroad activities indeed have the potential to foster 
the development of IC (Anderson & Lawton, 2011; Chieffo & Griffith, 2004; Williams, 2005). In Kehl 
& Morris (2008) study students who were enrolled in a long-term program (one semester) rather than 
a short-term program (8 weeks or less) also exhibited significantly higher global-mindedness than 
students on the home campus who plan to study abroad in the future. The fact that many studies 
found evidence for intercultural gains resulting from study abroad experiences, despite using quite 
different measurement instruments, argues rather persuasively for the transformational experience of 
study abroad activities. 

Theoretical underpinnings 

Intercultural competence (IC) is a complex, multi-layered and, therefore, also somewhat vague 
concept. It is not surprising then, that different definitions, terms, models, and operationalizations of 
the concept exist. A definition which could be shown to be deemed appropriate and useful by a panel 
of research experts and administrators of intercultural programs states that IC is “the ability to 
communicate effectively and appropriately in intercultural situations based on one’s intercultural 
knowledge, skills and attitudes” (Deardorff, 2006, p.247). As Deardoff points out this definition is 
derived from Byram’s (1997) work on intercultural communicative competence (ICC) and it will be 
used as a working definition in this paper. 

There are numerous models of IC. Some of them conceptualize IC mainly as a sum of different 
personality traits such as respect, empathy, flexibility, curiosity, tolerance of ambiguity, etc. (Ruben, 
1976). Other models, such as Byram’s model (1997) focus on different dimensions of IC (attitudes, 
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knowledge, awareness, skills). Again other models are mainly concerned with the developmental 
character of IC, the best-known among them being Bennett’s (1993) developmental model of 
intercultural sensitivity with its six developmental stages of Denial, Defense, Minimization, 
Acceptance, Adaptation, and, finally. Integration. 

All these theoretical concepts also have their limitations. As Leiprecht (2001, pp. 15-18) points 
out, general personality traits, such as empathy, tolerance of ambiguity, or flexibility may be necessary 
components of IC, but they are not enough to cover the construct. Firstly, these general personality 
traits have to be targeted at specific intercultural issues (e.g. empathy for perceived discrimination and 
marginalization) in order to function as constituting elements of IC. Secondly, IC also comprises other 
components such as behavioural components, cognitive components and value-based components 
(Leiprecht, 2001, p. 18). These feature in multi-dimensional constructs, such as the model of 
intercultural communicative competence by Byram. A multi-dimensional model, however, raises the 
question as to how closely related these different dimensions are and to what extent they can be 
considered constituting elements of a common underlying competence if they are only distantly 
related. With developmental models of IC the main problem consists in the fact that the different 
developmental stages were developed theory-driven and up to date, there is no adequate empirical 
validation of the different stages and their chronology. It is not clear whether all students really go 
through all of the proposed stages, whether everyone goes through these stages in the same order 
irrespective of the learning context, origin or age and if the proposed sequence of stages is really 
appropriate. 

Naturally, different models of IC also yield different operationalizations or approaches to 
measurement. There are numerous instruments for the assessment of IC, such as, for example, the 
Intercultural Adjustment Potential Scale (ICAPS) (Savicki, et al., 2004), the Assessment of 
Intercultural Competence (AIC) (Fantini & Tirmizi, 2006), the instruments from the Intercultural 
Competence Assessment Project (INCA) (Prechtl & Lund, 2007), the Intercultural Development 
Inventory (IDI) (Bennett & Hammer, 1998), the Multicultural Personality Questionnaire (MPQ) (Van 
der Zee & Van Oudenhoven, 2000) or the self-assessment tool Lolipop 1 Portfolio (Crosbie & 
Sudhershan, 2009). 

For our own operationalization of the construct of IC we used Byram’s (1997) model of 
intercultural communicative competence as a starting point. This model involves five elements or 
‘savoirs’. The following four elements were considered in our study: 

1. Knowledge ( savoir) 

2. Critical Cultural Awareness {savoir s’engager) 

3. Attitudes (savoir etre) 

4. Skills of discovery and interaction ( savoir cpprendre / faire) 

We did not include skills of interpreting and relating (. savoir comprendre ) since this revolves around 
text-based learning and working with documents which is more relevant for classroom learning. While 


1 Language on-Line Portfolio Project 
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Byram’s model integrates foreign language skills in the element ‘skills of discovery and interaction’ we 
treated this as a separate dimension since not all intercultural encounters also involve different 
languages. 

The Study 

The present study was carried out in Switzerland among upper secondary school students, many 
of whom are required to spend some time in a foreign language area as part of their educational 
degree. As Coleman (1998, p. 174) points out, ‘study abroad’ is not the most common term used in 
the European context to refer to this kind of experience. He uses the term ‘residence abroad’ in his 
outline of the European perspective. Neither term is entirely appropriate for the present study, though, 
since it includes both international (TL English) and intra-national stays (TL German or French). 
While all the participants stayed in an area that is linguistically and culturally different from their place 
of residence, not all of them went abroad. The multilingual nature of Switzerland with its four national 
languages offers ideal opportunities for contact with foreign language speakers within its borders. 
Consequendy, we will use the term ‘linguistic exchange activities’ to talk about the programs in which 
the participants of this study took place. 

The study examined a broad range of linguistic exchange activities at Swiss upper secondary 
schools which differ with respect to some important features mentioned above (duration, 
accommodation, affective and linguistic starting conditions, provisions for cultural interaction, etc.) in 
order to answer the following research questions: 

1. What effects do exchange activities have on the development of intercultural competence? 

2. What criteria for success can be identified? 

The study comprises different formats of exchange activities (exchange with a partner class, 
traditional language stay with attendance of a language school, internships, culture week, etc.) with 
different TLs. As upper secondary school students, the participants can be considered having 
intermediate to upper-intermediate TL competence. 

Sample and Design 

The study employed a quasi-experimental longitudinal design with an intervention group 
consisting of students who took part in an exchange program and a control group consisting of 
students who did not take part in an exchange program. Both groups were surveyed three times. The 
students in the intervention group were surveyed before going away (pretest), shortly after the return 
(posttest) and approximately three months after the return (delayed posttest). The control group was 
surveyed at the same intervals as the majority of the students who participated in an exchange 
program. This design allows inferences about both short-term and longer-term effects of linguistic 
exchange activities. 

The sample for the analysis of short-term effects of linguistic exchange activities consisted of 
405 upper secondary students from German and French-speaking Switzerland, who participated in an 
exchange program or went on a language stay (intervention group) and 135 upper secondary students 
from German and French-speaking Switzerland, who did not participate in an exchange program and 
did not go on a language stay (control group). The students were between 13 and 20 years-old. The 
average age was 16 years. Sixty-nine percent were girls and 31% boys. As far as the nationality of the 
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participants is concerned, 73% were Swiss, 18% double citizens and 10% foreigners. Fifty-seven 
percent were monolingual and 43% multilingual. 

The majority of the participants have a well-educated family background. More than half of the 
participants come from a family where at least one parent has a university degree. At least one of the 
parents of another 13% has an upper secondary school-leaving certificate 2 as the highest degree. 
Twenty-four percent indicate the vocational school certificate as the highest degree among their 
parents and 5% a secondary school certificate. Six percent of the participants did not know what the 
highest degree of their parents is. 

Table 1 provides an overview of the samples for the posttest (shortly after the return for the 
intervention group, four to six weeks after the pretest for the control group) 3 . 


Table 1: Sample Posttest 



Intervention group (n=405) 

Control group (n=135) 

Target language 

German or French (n=224) 

English (n=181) 

n=135 

Duration 

< 1 Month 

N=164 

> 1 Month 

N=60 

< 1 Month 

N=132 

> 1 Month 

N=49 



In the following we will characterize the exchange parameters of the students in the intervention 
group in more detail. 

Exchange activities in the traditional sense (with a partner class in the other linguistic/cultural 
area and bilateral exchange) constitute the minority in our sample. Most participants in our study stem 
from upper secondary schools which prescribe a three to four week stay in a foreign language 
(henceforth FL) area for certain classes (e.g. all eighth graders or all bilingual classes). The organization 
of this stay is usually up to the students with the requirement that no more than two students from 
the same school go to the same place. The students then look for a language school or an internship 
themselves. In terms of Engle & Engle’s (2003, pp. 10-13) classification of study abroad programs, 
the majority of the participants in our study took part in something that is most similar to the ‘short¬ 
term study’, with the exception that all course work was done in the TL and that the typical type of 
accommodation was an integrated home stay. 4 A minority of the participants who stayed for a whole 
semester also participated in something that is close to the cross-cultural immersion program. Quite 
a few upper secondary schools also offer one-week culture weeks where the entire class lives in a 
foreign culture for one week and carries out some project work. These latter probably correspond to 
the Study Tour in Engle & Engle’s (2003) classification. The fact that many upper secondary schools 
prescribe a FL stay for their students means that most of the participants in our study had to 
participate in an exchange activity and that not all of them were enthusiastic about this. This guarantees 
a certain heterogeneity in terms of the sample. 


2 This certificate qualifies for university entrance. 

’ This is the sample upon which the analyses of the short-term effects are based. 

4 See Coleman (1998, p. 174) for an elaboration of some basic differences between the 
prototypical American and European study abroad programs. 
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Almost two thirds (61%) of the students in the intervention group have attended a language 
school, 17% did an internship or voluntary work. Of the fourth (26%) that indicated having 
participated in another type of program, most participated in a traditional exchange with a partner 
school or partner class in the target area or they attended a regular upper secondary school in the 
target area. Besides this, the culture-week and au-pair work were mentioned as types of programs by 
a few participants. 

In terms of duration, the exchange activities studied here can be characterized as short-term 
activities for the most part. Thirty-seven percent of the intervention group participant spent one to 
two weeks in the target culture, another 38% three to four weeks, 13% five to six weeks and 7% each 
were away for seven to twelve weeks or for more than twelve weeks. Due to a time frame of two years 
set for the project, it was not possible to longitudinally evaluate students who spent a whole year in 
another linguistic/cultural area. There are, consequently, no students in the sample, who participated 
in a one-year program. 

As far as accommodation is concerned, the vast majority (87%) of the participants stayed in a 
host family (home stays). The remaining 13% stayed in a hotel/youth hostel, a dormitory, a rented 
apartment, a farm or at a relative’s /acquaintance’s place. 

The students in the control group stem from upper secondary schools whose curriculum does 
not include linguistic exchange activities or from upper secondary schools who only prescribe a foreign 
language stay for some of their classes. Students in this group, who reported previous participation in 
an exchange program, were eliminated from the analyses. 

At the time of the delayed posttest the sample has decreased by 36 participants in the control 
group and 65 participants in the intervention group. This is due to decreasing motivation on the part 
of the participants to fill in the questionnaire a third time, technical problems with the online tool and 
lastly, participants leaving the school. 

Instruments and data collection 

An online survey on their IC was administered to the participants three times between December 
2012 and October 2013. Information on important parameters of the exchange, including the kind 
of activities undertaken during the exchange, amount of contact with TL speakers, use of the TL 
during the exchange and type of accommodation, was only collected in the posttest (after the 
exchange). 

The instrument for the assessment of IC was developed by the research team themselves. Many 
items were inspired by a self-assessment tool of IC called the Lolipop 5 Portfolio (Crosbie & 
Sudhershan, 2009). This is an online language learning portfolio that was developed analogous to the 
European language portfolio. Unlike the European language portfolio it contains a fully integrated 
intercultural component with a set of unique ‘can-do’ statements for the self-assessment of IC. The 
can-do statements are based on two different theoretical models: Byram’s model of intercultural 
communicative competence, on the one hand and Bennett’s model of the development of 
intercultural sensitivity, on the other hand. While the can-do statements reflect the five different 


5 Language on-Une Portfolio Project. 
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‘savoirs’ in Byram’s model, they are also assigned to six different developmental levels which reflect 
the stages in Bennett’s model. In line with the European language portfolio these six different stages 
are labelled Al, A2, Bl, B2, Cl and C2. 6 

While the can-do statements of the Lolipop Portfolio were an important source of inspiration 
for the development of our own questionnaire, they were all heavily adapted for our purposes 
(questionnaire rather than portfolio, cultural target group known, etc.). Firstly, rather than opting for 
a model which a priori defines certain developmental stages and gauging development by means of 
items which represent certain developmental stages, we decided to assess different dimensions of IC 
longitudinally. We assume that the amount of intercultural development will be reflected in the amount 
of the respondents’ agreement/disagreement with items across time. 

Another important adaptation concerns the specificity of the items. Most measurement 
instruments of IC use culture-generic and, therefore, somewhat unspecific items (e.g. tries to 
understand other people’s behaviour) to assess IC. This is problematic to the extent that we cannot 
know what people or what cultural group(s) respondents are thinking of when responding to such 
items. As we do not know to what extent IC is a generic competence that an individual possesses 
irrespective of the target group or target culture or to what extent it is culture-specific this is an 
important stumbling block. It seems plausible that someone who may be interculturally competent 
when dealing with French, for example, is not necessarily also interculturally competent when dealing 
with Chinese, especially if we consider knowledge, intergroup attitudes, and language proficiency as 
constituting part of a person’s IC. 

With this in mind, we decided to formulate our items culture-/group-specific, so that we would 
know what cultural groups the students had in mind when responding to the questionnaire items. On 
the basis of the destination of most potential participants in our study we focused on the following 
cultural target groups: Americans, English, Canadians, Australians, French-speaking Swiss, French, 
German-speaking Swiss, and Germans. The students in the intervention group got a questionnaire 
with items targeted at the cultural group with which they had contact during their exchange activity. 
For the students in the control group, who were not in a contact situation with any other cultural 
group, one of the above-mentioned groups was chosen at random. As a result the questionnaire items 
certainly had a somewhat different meaning for the participants in the control group than for those 
in the intervention group. 

The questionnaire was developed in the online questionnaire tool Survalyzer and piloted in four 
school classes (two classes in the control group and two classes in the intervention group). On the 
basis of the comments from the pilot classes and the experiences during the piloting, the questionnaire 
was subsequently revised. The questionnaire was developed in German and translated into French by 
a translation bureau. 

For the assessment of the participants’ intercultural experiences so far and important information 
on their exchange program we based ourselves on instruments of INCA (Prechtl & Lund, 2007), 
MacFarlane (1997) and Ogay (2000). 


6 For more information on the Lolipop Portfolio, see Crosbie and Sudhershan (2009). 
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Analysis 

Scaling and grouping of questionnaire data. 

For the construct ‘intercultural competence’, scales were generated by means of Rasch analysis 
where possible. As all items were polytomous with four response categories, the partial credit Rasch 
model was used (Masters, 1982). The scales were constructed with the data from the pretest. Later 
the same scaling procedure was used for the posttests in order to make longitudinal comparisons 
possible. The software RUMM2030 was used to conduct theses analyses (Andrich, et al., 2010). 

The dimension, self-assessed language competence will not be analysed further in this article. The 
dimension knowledge included questions on the participants’ knowledge about cultural similarities and 
differences as well as intercultural communication on the one hand (eight items) and questions about 
their awareness about their own linguistic and intercultural learning process (four items) on the other 
hand. No Rasch scale could be constructed for the latter. For the former a Rasch scale could be 
generated but the scale has a low person-separation index of 0.45 which means that the participants’ 
knowledge is not estimated in a reliable fashion and that the scale does not reliably distinguish between 
students with a lot of intercultural knowledge and students with little intercultural knowledge. For that 
reason the dimension knowledge will not be used as a target variable in the ensuing analyses. 

The dimension skills comprised 14 items about handling cultural diversity and linguistic as well as 
cultural challenges. Four items had to be eliminated due to large fit-residuals. The remaining items 
yielded a one-dimensional Rasch scale. The item “I can compare the culture (e.g. eating habits, 
clothing, family life, school life, work life, etc.) of x and y and detect similarities and differences” 
showed differential item functioning (DIF). Participants who went to a German-speaking area (that is to 
say French-speaking Swiss) consider themselves to be better at this than participants who went to a 
French or English-speaking area. Furthermore, there was no clear distinction between the answer 
categories ‘not true’ and ‘rather not true’. Therefore, these two categories were merged. There was 
also DIF with regard to the item “I observe how the English behave in different situations and through 
this I learn more about their habits and way of living”. Females have higher values on this which 
means that they consider themselves to do this more than males. Both these items were separately 
calibrated for the respective groups for the construction of the Rasch scale. 

With regard to the dimension attitudes we distinguished two different subdimensions. The first 
dimension is comprised of items which primarily target the participants’ willingness to engage with 
people from another cultural and linguistic background and to engage with the intercultural learning 
process associated with this (willingness, seven items). The second dimension is comprised of items 
that deal with the feelings that the participants have towards the target area and TL speakers (affect, 
twelve items). There were too many ill-fitting items and too much differential item functioning (DIF) 
in both categories for the construction of a Rasch scale. Therefore, a factor analysis was carried out 
with all the items (Principal axis analysis, Varimax rotation). The measure of sampling adequacy 
(Kaiser-Meyer-Olkin) was .83. Three items (all of them negatively worded) were eliminated because 
they had low communalities and correlations with the other items. The scree plot indicated one or 
three factors. Except for one item (“I try not to judge x prematurely”) the willingness items all clearly 
loaded on one factor. The one ill-fitting item was eliminated and a scale was constructed with all the 
remaining willingness items (attitudes -willingness). Cronbach alpha is .77 and the corrected item- 
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total correlation ranges from .39 to .68. With the affect items the mapping was less clear. A factor 
analysis of the affect items alone yields a measure of sampling adequacy of .80 and the scree plot 
points to two factors. Consequently, two affect scales were generated. Table 2 provides an overview 
of the scales and their psychometric properties. 


Table 2: Scales and their psychometric properties 


Rasch Scales 

No. of 

Items 

Person- 

separation Index 

Self-assessed language 
competence 

14 

.86 

IC - Knowledge 

7 

.45 

IC-Skills 

10 

.77 


Other Scales 

No. of Items 

Cronbach Alpha 

Factor loadings 

Item-total correlation 

Total Variance explained 

IC - Attitudes: 
Willingness 

5 

.77 

.37 bis .83 

.39 bis .68. 

53.9% 

IC - Atittudes: Affect 1 

6 

.82 

.41 bis .81 

.45 bis .69 

IC - Atittudes: Affect 2 

4 

.71 

.54 bis .74 

.44 bis.59 



Apart from the questions revolving around the students’ IC, the questionnaire also contained 
questions about the language classes at the home school as well as the exchange program as such. The 
students were asked, for example, what aspects are important to their foreign language teacher in the 
foreign language classes (grammar, vocabulary, communication or pronunciation) or in what kind of 
activities they participated during their stay abroad (e.g., going to the movies, sightseeing, participating 
in a club, etc.), who they talked to in which language, etc. This data was examined by means of 
descriptive statistics as well as cluster analysis in order to detect patterns and generate variables on this 
basis. 

Analysis of covariance (ANCOVAs) 

For each target variable (each dimension of IC) two separate analyses of covariances (ANCOVA) 
were carried out for the posttest and the delayed posttest. The first ANCOVA included both the 
intervention and the control group. The main aim was to determine whether it plays a role for the 
learners’ IC if they participated in an exchange activity or not. The second ANCOVA only involved 
the intervention group and analyzed which demographic, school-related and study abroad-related 
variables contribute to the explanation of the students’ IC at the time of the posttest and delayed 
posttest. The values of the target variable at the time of the pretest were considered as covariables for 
these analyses. In this way, the participants’ starting conditions were controlled for. 

The analyses of the posttest data demonstrate what short-term effects exchange activities have 
(short-term effect). The analyses of the delayed posttest data demonstrate what effects they have in 
the longer run (long-term effects). The following variables were considered as possible explanatory 
variables: gender, parents’ educational background, linguistic background, nationality, age, previous 
stays abroad, previous exchange activities, previous residence in TL area, type of program, 
preparation, accommodation, duration, contact with TL speakers, use of TL, activities undertaken 
during the stay, location. 

For the posttest data the variables were selected on the basis of the p-value by means of 
backwards stepwise elimination. For the delayed posttest data, a more elaborate analysis technique was 
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employed. The variables were selected on the basis of the Akaike information criterion (AIC). Since 
this usually yields models that are too comprehensive including non-significant variables, the BIC 
criterion was subsequently used to continue the variable selection. Furthermore, a ten-fold forward 
cross-validation was used to prevent overfitting and to get a more realistic estimation of prediction 
errors (Hastie, et al., 2009). 

The analyses were done in SPSS 21 und R 3.0.2. 

Results 

Short-term effect of exchange activities 

Willingness to engage with people from another cultural background. 

The ANCOVA of the posttest data across all participants showed that the learners’ willingness 
to engage with people from another cultural and linguistic background is largely dependent on the 
extent to which they were willing to do so even before the exchange (see Table 3). Participating in an 
exchange activity also seems to affect learners’ willingness to engage with people from another cultural 
background since students in the intervention group exhibited higher willingness than those in the 
control group. The same goes for students whose parents have a tertiary education. Furthermore, 
there is an interaction between teaching methodology and the linguistic background of the students. 
If, in the eyes of the students, the home teacher does not emphasize communication in the foreign 
language lessons, monolingual students are significantly less willing to engage with people from 
another cultural and linguistic background than multilingual students. If the teacher emphasizes 
communication, monolingual and multilingual students do not differ. It is primarily the monolingual 
students, consequently, whose willingness to engage with people from other cultural and linguistic 
backgrounds profits from foreign language lessons that emphasize communication and interaction. 


Table 3: Short-term effects on willingness (intervention and control group) 



Regressions coefficient B 

P 

Intercept 

4.571 

.000 

Control group 

-1.609 

.000 

Willingness pretest 

.612 

.000 

Parents' educational background: no tertiary education 

-.981 

.000 

monolingual 

-.154 

.528 

FL lessons: communication unimportant 

.391 

.490 

Linguistic background*FL lessons 

-1.754 

.027 


R 2 =.50 


The analysis of the intervention group alone yields a similar picture. The best predictor of the 
participants’ willingness to engage with people from the TL area shortly after their return home, is 
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their willingness to do so before departure. Students whose parents have a tertiary education and 
multilingual students exhibit a stronger willingness to engage with people from the TL area. Besides 
this, some exchange-related variables could also be shown to play a role. The more often the 
participants used the TL during their stay, the higher their willingness to engage with people from the 
TL area after their return. Similarly, this willingness is higher among participants who reported having 
spoken regularly to the teachers of the partner school/language school than among those who 
reported doing this irregularly or never (see Table 4). 


Table 4: Short-term effects on willingness (intervention group) 



Regression coefficient B 

P 

Intercept 

3.501 

.000 

Willingness T1 

.558 

.000 

Parents' educational background: no tertiary education 

-.951 

.000 

monolingual 

-.524 

.045 

Use of TL 

.030 

.000 

No contact with teachers of partner / language school 

-.830 

.004 


R2=.45 


Interculturol attitudes: affect. 

The more positive the students’ attitudes towards representatives of the host culture were at the 
time of the pretest, the more positive they also were at the time of the posttest. So, once again the 
students’ starting conditions play a crucial role. Besides this, the interaction between group 
membership (intervention or control group) and the parents’ educational background plays a role (see 
Table 5). While in the control group students whose parents have a tertiary education are characterized 
by significantly more positive attitudes towards representatives of the host culture, the educational 
background of the parents does not play a role in the intervention group. It seems, consequently, that, 
in particular, students whose parents do not have a tertiary degree profit from exchange activities in 
terms of attitudes. 


Table 5: Short-term effects on intercultural attitudes (intervention and control group) 


Regression coefficient B p 



5.265 

.000 

Intercept 

Control group 

-.613 

.205 

Attitudes: Affect pretest 

.629 

.000 

Parents' educational background: no tertiary education 

-.545 

.079 

Group membership (intervention vs control 
group)*Parents' educational background 

-2.076 

.021 


R2=.48 


©2015 The Forum on Education Abroad 


198 



Frontiers: The Interdisciplinary Journal of Study Abroad 


Volume XXV, Fall 2015 


In the analysis of the intervention group alone, the affective starting conditions of the participants 
also emerged as the most important explanatory factor. In addition, the amount of use of the TL and 
regular contact with TL speakers were once again relevant. The more the students used the TL during 
their stay, the more positively disposed they are towards representatives of the host culture after their 
return. Regular contact with work colleagues or superiors has a positive effect on the students’ 
intercultural attitudes. Finally, it turned out to be relevant for the students’ intercultural attitudes what 
kind of study abroad program the students were involved in. Those students who did an internship 
or voluntary work during their stay had less positive intercultural attitudes after their return than those 
who participated in another kind of program (attending a partner/language school, working as an au- 
pair, etc.) (see Table 6). 


Table 6: Short-term effects on intercultural attitudes (intervention group) 



Regression coefficient B 

P 

Intercept 

3.324 

.000 

Attitudes: Affect pretest 

.667 

.000 

Kind of program: no internship /no voluntary work 

1.211 

.009 

No contact with work colleagues / superiors 

-1.118 

.005 


R 2 =.42 


Intercultural skills. 

With regard to the intercultural skills of the participants, the analysis of the posttest data also 
shows that it makes a difference whether students participated in an exchange activity or not. The 
students in the intervention group are interculturally significantly more skilled than those in the control 
group. Gender also plays a role here. In the control group males report significandy higher intercultural 
skills than females. In the intervention group males and females differ less and it is the females who 
report higher intercultural skills. Once more, the starting conditions of the participants play a crucial 
role since the intercultural skills at the time of the pretest turn out to be the best predictor of the 
intercultural skills at the time of the posttest (see Table 7). 


Table 7: Short-term effects on intercultural skills (intervention and control group) 



Regression coefficient B 

P 

Intercept 

.697 

.000 

Control group 

-.859 

.000 

Intercultural skills pretest 

.590 

.000 

Girls 

.217 

.092 

Group membership (intervention vs control group)*Gender 

-.977 

.000 


R 2 =.55 


A look at the intervention group alone reveals that apart from the participants’ starting conditions a 


©2015 The Forum on Education Abroad 


199 



Sybille Heinzmann, Roland KOnzIe, Nicole Schallhart, & Marianne Muller 


number of exchange-related and teaching-related variables play a role. The more the students who 
participated in an exchange activity report having used the TL during their stay, the higher their 
intercultural skills shordy after their return. The duration of the stay also plays a role. A longer stay 
tends to have a positive effect. Those participants who only stayed in the host culture for one to two 
weeks exhibit the lowest values. This group only differs significantly from the group of students who 
stayed in the host culture for five to six weeks, however. Somewhat surprisingly, this latter group had 
the highest intercultural skills, even though they did not significantly differ from the group of students 
who was away for more than six weeks. Consequently, we can see a U-shaped development here where 
students with a medium length of stay seem to profit most and both students with a shorter and 
longer duration of stay are characterized by lower intercultural skills. 

Once again regular contact with the teacher(s) of the partner school or language school also 
exerts a positive influence. But the teacher(s) at the local school also influence the intercultural skills 
of the exchange students. Students whose local FL teachers attach importance to communication and 
grammar have higher intercultural skills. Furthermore, the females in the intervention group report 
significantly higher intercultural skills then the males (see Table 8). 


Table 8: Short-term effects on intercultural skills (intervention group) 



Regression coefficient B 

P 

Intercept 

.182 

.507 

Skills pretest 

.541 

.000 

Use of TL 

.010 

.000 

No contact with teachers of partner / language school 

-.518 

.000 

Duration of stay: 1-2 weeks 

-.172 

.344 

Duration of stay: 3-4 weeks 

.087 

.621 

Duration of stay: 5-6 weeks 

.314 

.143 

FL lessons: communication unimportant 

-.438 

.015 

FL lessons: grammar unimportant 

-.469 

.028 

Girls 

.321 

.009 


R 2 =.35 


Long-term effect of exchange activities 

In the last section we saw that students who left their home to work or study in another cultural 
environment for some time are significantly more interculturally competent upon their return than 
students who stayed at home. In this section we will analyse whether this positive effect of exchange 
activities lasts fairly long-term. 

Willingness to engage with people from another cultural background. 

At the time of the delayed posttest, the learners’ willingness to engage with people from another 


©2015 The Forum on Education Abroad 


200 



Frontiers: The Interdisciplinary Journal of Study Abroad 


Volume XXV, Fall 2015 


cultural and linguistic background and to learn something about them and their culture is still largely 
dependent on the extent to which they were willing to do so before the exchange. Still, the students 
who participated in an exchange are more willing to engage with people from another cultural and 
linguistic background than those who did not (see Table 9). 


Table 9: Long-term effects on willingness (intervention and control group) 



Regression coefficient B 

P 

Intercept 

3.192 

.000 

Willingness pretest 

.576 

.000 

Intervention group 

1.022 

.002 

Parents' educational background: tertiary education 

.678 

.008 

Boys 

-.575 

.040 

Previous stays abroad: 0-2 times 

-1.557 

.004 

Previous stays abroad: 3-5 times 

-.390 

. 394 

Previous stays abroad: 6-10 times 

-.358 

.375 

Previous stays abroad: > 15 times 

-.123 

.748 

Age: 16 years 

-.032 

.922 

Age: > 16 years 

-.611 

.080 


R 2 =.44 


Apart from these two most dominant factors, a series of demographic variables also play a role. 
Students whose parents have a tertiary education and girls are significantly more willing to engage with 
people from another cultural background, students who are older than 16 years tend to be less willing 
to do so. Last but not least, the students’ willingness to interact with people from another cultural 
background also depends on the number of times they have been abroad. Students, who have never 
been abroad, or only once or twice, are significantly less willing to engage interculturally. 

A look at the intervention group alone confirms that the students’ starting conditions are the 
most powerful explanatory factor for their willingness to engage with people from another cultural 
background three months after their return home. This is followed by the number of previous stays 
abroad, with students who have never been abroad, or only once or twice scoring significantly lower 
than the others. Next to this, a number of exchange-related variables could also be shown to be 
significant. The longer the students stayed in the host culture, the higher their willingness to engage 
with representatives of that culture three months after their return. An intensive contact with local 
people during one’s stay also has a positive effect. Students who indicated having had weekly or daily 
contact with local students or work colleagues during their stay demonstrate a significantly higher 
willingness for intercultural engagement three months after their return than those who indicated 
having had less than weekly contact (see Table 10). 
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Table 10: Long-term effects on willingness (intervention group) 



Regression coefficient B 

P 

Intercept 

3.876 

.000 

Willingness pretest 

.480 

.000 

Duration of stay: 3-4 weeks 

.277 

.422 

Duration of stay: 5-6 weeks 

.554 

.317 

Duration of stay: more than 6 weeks 

1.732 

.000 

Previous stays abroad: 0-2 times 

-1.790 

.005 

Previous stays abroad: 3-5 times 

-.178 

733 

Previous stays abroad: 6-10 times 

-.366 

.443 

Previous stays abroad: > 15 times 

-.014 

.974 

Regular contact with local students or work colleagues 

.920 

.009 


R 2 =.39 


Interculturol attitudes: affect. 

How positively the students are disposed towards representatives of the TL culture three months 
after their return and how comfortable they feel among them largely depends on their affective 
dispositions before their departure. Next to this, the exchange experience itself also has an effect. The 
students in the intervention group are significantly more positively disposed towards representatives 
of the target culture than students in the control group and they feel significantly more comfortable 
among them (see Table 11). 


Table 11: Long-term effects on intercultural attitudes (intervention and control group) 



Regression coefficient B 

P 

Intercept 

1.593 

.001 

Intercultural attitudes pretest 

.737 

.000 

Intervention group 

1.974 

.000 


R 2 =.53 


The ANCOVA which only includes the intervention group demonstrates that, besides the 
affective starting conditions, the duration of stay has an influence on the students’ intercultural 
attitudes three months after their return. The longer the stay in the target culture, the more positive 
the students’ attitudes towards representatives of that cultural group are (see Table 12). 


Table 12: Long-term effects on intercultural attitudes (intervention group) 



Regression coefficient B 

P 

Intercept 

3.538 

.000 

Intercultural attitudes pretest 

.694 

.000 

Duration of stay: 3-4 weeks 

.624 

.075 

Duration of stay: 5-6 weeks 

1.094 

.049 

Duration of stay: > 6 weeks 

1.546 

.002 


R 2 =.46 
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Interculturol skills. 

With regard to the students’ intercultural skills, the analysis across the intervention and the control 
group once again illustrates that participating in an exchange activity makes a difference not only short¬ 
term but also long-term. At the time of the delayed posttest the students in the intervention group 
are interculturally significantly more skilled than the students in the control group. The most powerful 
explanatory variable, however, is once again the student’s starting conditions, that is, their intercultural 
skills at the time of the pretest (see Table 13). 


Table 13: Long-term effects on intercultural skills (intervention and control group) 



Regression coefficient B 

P 

Intercept 

-.351 

.013 

Intercultural skills pretest 

.584 

.000 

Intervention group 

1.051 

.000 


R 2 =.48 


When looking at the intervention group in isolation, only the students’ intercultural skills before 
the exchange could be shown to significantly predict their intercultural skill three months after their 
return (see Table 14). 


Table 14: Long-term effects on intercultural skills (intervention group) 



Regression coefficient B 

P 

Intercept 

-.351 

.013 

Intercultural skills pretest 

.543 

.000 


R 2 =.26 


Discussion and Conclusions 

The analyses across the intervention and control group consistently indicate that a stay in an area 
that is culturally and linguistically different from one’s own has a positive influence on the development 
of one’s IC. The most consistent finding in this study was that the students in the intervention group 
are interculturally more competent than those in the control group. They are more willing to engage 
with representatives of the target culture and to learn from and with them. They are also more 
positively disposed towards representatives of the target culture and have higher skills of discovery 
and interaction. Unlike in the study by Rexeisen, et al. (2008) this is not only true of the time shortly 
after their return home but seems to be a lasting asset as the analyses of the delayed posttest data 
show. Given that the vast majority of the exchange activities studied here were short term activities 
(one to six weeks), the study reinforces findings from other studies (Anderson, et al., 2006; Chieffo & 
Griffith, 2004; Dwyer, 2004; Donnelly-Smith, 2009) which also demonstrate positive effects of short¬ 
term programs on intercultural learning. This is an encouraging finding, particularly in light of the fact 
that short-term study abroad programs clearly dominate the field due to their more modest demands 
in terms of prior linguistic and cultural preparation, as well as financial resources (Engle & Engle, 
2003). Provided a number of factors that influence intercultural learning (see discussion below) are 
taken into account, short-term programs can be more than the Vacation’ as which they are sometimes 
decried (Donnelly-Smith, 2009). 

As expected, the analyses also show that an exchange activity per se and of itself is not the key 
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to successful intercultural development. The extent to which the participants can profit from an 
exchange experience depends on a number of factors. The affective starting conditions of the 
participants seem to be a major factor. The IC of the participants before the exchange always 
constituted the best predictor of their IC after the exchange. This is not surprising, as you would 
expect that students who are interculturally more competent before the exchange are also more 
competent after the exchange. Given the central role that students’ starting conditions seem to play in 
their further development, it seems advisable to raise the students’ awareness for their own 
expectations, attitudes and needs before they embark on an exchange. Raising awareness for their own 
starting conditions facilitates working on them and identifying areas for improvement. It is only once 
students are conscious of attitudes and expectations that negative attitudes can be improved and 
unrealistic expectations relativized. Some of the students’ comments on the questionnaire suggest that 
there was little reflection on their part about their intercultural attitudes and skills. As one student put 
it “I never asked myself these questions before”. 7 Another student said: “Actually I haven’t given much 
thought to this yet”. 8 It seems worthwhile that schools intensify student preparation in this area. 
Somewhat surprisingly, student preparation could not be shown to affect the participants’ intercultural 
development. The questions relating to student preparation in the questionnaire focused on the 
provision of information on the place where the students would stay, on communicative norms and 
effective communication strategies and on potential communicative or cultural challenges. It is 
possible that the preparation of students for their intercultural experience should not necessarily or 
not exclusively focus on the provision of geographical, historical, cultural and linguistic information 
but rather on the students’ expectations, attitudes and pre-existing skills. 

While the results of our study suggest that even rather short exchange activities positively affect 
student’s IC, they also show that the duration of stay is an important factor influencing ultimate 
success. With the exception of the students’ intercultural skills, where a U-shaped development was 
identified, the data suggests that the longer the better. The longer the students stay in the target culture 
area, the more they seem to grow interculturally. This is a finding that is reinforced by many of the 
student comments in the questionnaire who lament that more time would be needed to really improve 
one’s language and intercultural skills. The U-shaped developmental pattern found for the students’ 
intercultural skills where students with a medium length of stay seem to profit most and both students 
with a shorter and longer duration of stay are characterized by lower intercultural skills also reflects 
previous research. It can potentially be accounted for by the fact that people who only stay in the host 
culture for a very short time have extremely limited opportunities to substantially develop 
interculturally. People who stay among people from the host culture for a longer time have more 
opportunities to really delve into everyday life there and encounter challenges or frustrations. While it 
is exactly such challenges or frustrations that forces study abroad students to develop interculturally 
(Ogay, 2000, pp. 274-277), they may also lead to a more cautious and, hence, lower assessment of their 
IC on the part of these students. What the findings certainly suggest is that a stay of one to two weeks 
is not particularly useful for the development of IC; since especially the group of participants who 
went on a one-to-two week exchange were less interculturally competent than the other groups. This 


7 Translation by the authors. Original: "Au paravant je me suis jamais pose ces questions. ” 

8 Translation by the authors. Original: "Eigentlich habe ich mir noch nicht so viele Gedanken 
gemacht.” 
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does not necessarily mean that several such rather short stays cannot have a long-term effect. Such 
short programs may also function as a door opener for further and longer studies abroad (Chieffo & 
Griffith, 2004; Engle & Engle, 2003). Nevertheless, exchange program coordinators/administrators 
should try to ensure a suitable duration of stay. 

Furthermore, exchange programs or study abroad programs should foster regular contact with 
local people and promote active use of the TL through organisational structures (for example, living 
with a host family or individual exchange activities) as these two features have also proved relevant for 
the participants’ IC following their return (Engle & Engle, 2004, p.232). It seems likely that these two 
variables are indicative of a focused and reflective interaction with the host culture which is what 
ultimately separates a study abroad program from study at home and what most distinguishes different 
study abroad programs (Engle & Engle, 2003, p. 4). A focused and reflective interaction with the host 
culture can be regarded as experiential learning where students truly experience the culture in which 
they reside (Anderson & Lawton, 2011, p. 87). It may even be advisable to integrate such experiential 
learning components (e.g. local tandem partner, community service) as a compulsory element of 
exchange programs as in the AUCP program described by Engle and Engle (2004). 

Limitations 

The most notable limitation of this study is probably the fact that the questions on IC in our 
questionnaire were not equally meaningful for the intervention and the control group. Our decision 
to formulate group-specific items had the advantage that we would be sure that students in the 
intervention group would really report on their perceptions of and attitudes towards the group with 
which they were in contact. It also meant, however, that the students in the control group would also 
answer questions in relation to one particular group (e.g. Swiss Germans) which was randomly 
assigned. Consequently, they were answering questions about a group they had no contact with at that 
time and maybe never had contact with rather than answering questions in relation to foreigners in 
general. This very fact may have made some of the items difficult for them to answer as they would 
have to imagine how they would deal with people from that group, how much they like them or how 
comfortable they would feel among them. As far as the pretest is concerned, this is also true of the 
students in the control group since at that point they were not yet in contact with the group in 
question. Future research should investigate how closely related measures of a general IC and 
measures of a group-specific IC are and whether we can usefully assume that IC is a basic skill that is 
independent of the target group. 

Another drawback of questionnaires on IC and of the construct of IC in general is that they, by 
definition, imply that there are clearly defined groups of people that somehow differ in terms of 
cultural habits and norms. Questions on norms, values, and habits of certain cultural groups 
necessarily homogenize that group and treat them as distinct entities. This bears the danger of 
stereotyping. It is questionable to what extent such a homogeneity and separateness of culture can 
realistically be assumed in the face today’s extent of migration, increase of regional and global 
interconnectedness and the internal differentiation of modern societies where many people are 
multicultural and cannot be assigned to a specific culture. 

Despite these limitations, the present study provides useful insights for study abroad program 
administrators, teachers and students thanks to a number of methodological merits, most notably the 
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experimental design comprising an intervention and a control group, a pre-, post-, and delayed 
posttest, the large sample size, the fact that the participants were required to study abroad rather than 
choosing this option themselves, and the large range of types of programs that could be investigated. 
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